Probability theory refresher

We are all familiar with the phrase “the probability that a coin will land heads is 0.5”. But what does this mean? There are actually two different interpretations of probability. One is called the frequentist interpretation of probability. In this view, probabilities represent long run frequencies of events. For example, the above statement means that, if we flip the coin many times, we expect it to land heads about half the time. The other interpretation is called the Bayesian interpretation of probability. In this view, probabilities represent measures of uncertainty or degrees of belief [Jay03]. In the Bayesian view, the above statement means we think the coin is equally likely to land heads or tails on the next toss. One big advantage of the Bayesian interpretation is that it can be used to model our uncertainty about events that do not have long term frequencies. For example, we might want to compute the probability that the polar ice cap will melt by 2020AD. This event will happen zero or one times, but cannot happen repeatedly. Nevertheless, we ought to be able to quantify our uncertainty about this event; based on how probable we think this event is, we will make appropriate decisions/ take appropriate actions. We might also be interested in computing the probability of counterfactual events, such as the probability that the ice cap would have melted in 2000AD if the Kyoto protocol had not been ratified. Despite the two different philosophical interpretations, the mathematics of probability theory remains the same. We assume the reader is already familiar with the basic notions of probability and random variables, and simple descriptive statistics, such as the mean and variance. (For an excellent introduction, see e.g., [Was04, Ric95]). However, below we give a quick refresher. We then introduce a variety of probability distributions that will be used later.